Skip to content

[v2] Fix bug with auth_scheme_preference#10169

Open
aemous wants to merge 17 commits intoaws:v2from
aemous:bugfix-auth-scheme-pref
Open

[v2] Fix bug with auth_scheme_preference#10169
aemous wants to merge 17 commits intoaws:v2from
aemous:bugfix-auth-scheme-pref

Conversation

@aemous
Copy link
Copy Markdown
Contributor

@aemous aemous commented Mar 27, 2026

Notes:

  • Currently, auth_scheme_preference only works as intended for modeled auth schemes at the service level (e.g. bedrock). It does not work for auth being resolved from other points in the precedence chain (e.g. EP2.0, operations).
    • For example, for S3 Outposts, users cannot use auth_scheme_preference to choose sigv4 over sigv4a. Outposts models auth at the EP2.0 level, currently.

Description of changes:

  • auth_scheme_preference is now used to reprioritize auth schemes, if configured, if the service models auth schemes at the endpoint level.
  • auth_scheme_preference is now used to reprioritize auth schemes, if configured, if the service models auth schemes at the operation level.

Description of tests:

  • Successfully ran pre-prod build workflow.
  • Manually tested auth_scheme_preference shared config file using the following test cases:

Endpoint-level auth test

User configures sigv4 over sigv4a. We expect sigv4 to be used.

auth_scheme_preference = sigv4,sigv4a
aws s3api get-object --bucket arn:aws:s3-outposts:us-east-1:ACCOUNT:outpost/op-REDACTED/accesspoint/my-outpost-ap-2 --key test-object.txt --region us-east-1 output.txt --debug
2026-03-27 16:12:17,315 - MainThread - botocore.endpoint - DEBUG - Sending http request: 
<AWSPreparedRequest stream_output=True, 
method=GET, 
url=https://my-outpost-ap-2-ACCOUNT.op-REDACTED.s3-outposts.us-east-1.amazonaws.com/test-object.txt, 
headers={
'x-amz-checksum-mode': b'ENABLED', 
'User-Agent': b'aws-cli/2.33.13 md/awscrt#0.31.2 ua/2.1 os/macos#25.3.0 md/arch#arm64 lang/python#3.12.6 md/pyimpl#CPython m/n,E,b,Z cfg/retry-mode#standard md/installer#source md/prompt#off md/command#s3api.get-object', 
'X-Amz-Date': b'20260327T201217Z', 
'X-Amz-Security-Token': b'REDACTED', 
'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 
'Authorization': b'AWS4-HMAC-SHA256 Credential=REDACTED/us-east-1/s3-outposts/aws4_request, 
SignedHeaders=host;x-amz-checksum-mode;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=077ee48a7141356c705cbe03042ef71cb47de9b9e366e7f80021582c3fb62b53'}>

Result: AWS4-HMAC-SHA256 in the Authorization header value confirms sigv4 was used.


User configures sigv4a over sigv4. We expect sigv4a to be used.

auth_scheme_preference = sigv4a,sigv4
aws s3api get-object --bucket arn:aws:s3-outposts:us-east-1:ACCOUNT:outpost/op-REDACTED/accesspoint/my-outpost-ap-2 --key test-object.txt --region us-east-1 output.txt --debug
2026-03-27 16:17:07,382 - MainThread - botocore.endpoint - DEBUG - Sending http request: 
<AWSPreparedRequest stream_output=True, 
method=GET, 
url=https://my-outpost-ap-2-ACCOUNT.op-REDACTED.s3-outposts.us-east-1.amazonaws.com/test-object.txt, 
headers={
'x-amz-checksum-mode': b'ENABLED', 
'User-Agent': b'aws-cli/2.33.13 md/awscrt#0.31.2 ua/2.1 os/macos#25.3.0 md/arch#arm64 lang/python#3.12.6 md/pyimpl#CPython m/Z,E,n,S,b cfg/retry-mode#standard md/installer#source md/prompt#off md/command#s3api.get-object', 
'host': b'my-outpost-ap-2-ACCOUNT.op-REDACTED.s3-outposts.us-east-1.amazonaws.com', 
'X-Amz-Security-Token': b'', 
'X-Amz-Date': b'20260327T201707Z', 
'X-Amz-Region-Set': b'*', 
'x-amz-content-sha256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 
'Authorization': b'AWS4-ECDSA-P256-SHA256 Credential=REDACTED/s3-outposts/aws4_request, 
SignedHeaders=host;x-amz-checksum-mode;x-amz-content-sha256;x-amz-date;x-amz-region-set;x-amz-security-token, Signature=30450220429b7f10af468e9d806fcbd10a09a40b0f46ee26bc07881ea2eab3a4b9f90548022100a1578454f460ffc1f240fd85633ec5368a289e44072deff1ec2a747a7c5b7ccb'}>

Result: AWS4-ECDSA-P256-SHA256 in the Authorization header value confirms sigv4a was used.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@aemous aemous requested a review from SamRemis March 27, 2026 20:23
@aemous aemous marked this pull request as ready for review March 27, 2026 20:23
@aemous aemous added the v2 label Apr 2, 2026
s3_disable_express_session_auth = config_kwargs[
's3_disable_express_session_auth'
]
preferred_auth_schemes = config_kwargs['auth_scheme_preference']
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit, non-blocking]
IMO this variable name should exactly the kwarg name so there's no confusion. There are already a ton of variables around signature version. In this file alone:

signature_version -> sig_version -> requested_auth_scheme, then also you're now adding auth_scheme_preference -> preferred_auth_schemes. It's a lot of complexity to keep in your head, so while it doesn't make a huge difference, I'd prefer to see this renamed to just auth_scheme_preference to make it clear that it's the same thing as the config variable.


# if a preferred auth schemes list is provided, reorder the auth schemes
# list based on the preferred ordering.
if self._preferred_auth_schemes is not None:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Change requested]
We already have a function called resolve_auth_scheme_preference which handles this and should avoid duplicating preference-resolution / ordering logic here.

Right now we manually reorder the endpoint ruleset authSchemes list based on auth_scheme_preference. Instead, we can reuse resolve_auth_scheme_preference() to pick the desired auth type and then map that back to the actual ruleset scheme dict. Here's the proof of concept I wrote:

        if self._requested_auth_scheme is not None:
             ....
        elif self._preferred_auth_schemes is not None:
            prefs = self._preferred_auth_schemes.split(',')
            available_ruleset_names = [s['name'].split('#')[-1] for s in auth_schemes]
            auth_schemes_by_auth_type = {self._strip_sig_prefix(s['name'].split('#')[-1]): s for s in auth_schemes}
            resolved_auth_type = resolve_auth_scheme_preference(prefs, available_ruleset_names)
            name = resolved_auth_type
            scheme = auth_schemes_by_auth_type[resolved_auth_type]

We could definitely refactor this and/or the resolve_auth_scheme_preference to make it much cleaner. What do you think?

@@ -0,0 +1,5 @@
{
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High level note:

Jonathan has a semi-related bugfix that will likely need to apply here as well - https://github.com/boto/botocore/pull/3663/changes.

When we resolve v4a via auth scheme preference, we need to make sure we are also respecting the signing region set

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants